About Tomasz Tunguz

2024-05-17 08:00:00

Ramp published its quarterly spending trends & revealed how businesses are spending on AI. There are many great data points that underscore the growth in AI but there are important nuances in the patterns.

First, AI growth rates across the most popular vendors have fallen 78% annually. On average, these AI businesses are growing customer counts 105% ; the median is 38%.¹

However, this isn’t a uniform trend. Some companies have seen contraction in customer counts. Both the companies with negative account retention still see relatively minor account churn : 15-25% account churn at these price points is common.

Second, the account expansion remains relatively strong. The top quartile software companies around 125% today & the companies above are very similar with a median of 129% net dollar retention (NDR).

These data points suggest that AI infused software companies’ metrics will asymptote to overall software metrics in time. NDR is almost equal. Growth rates remain very strong, but perhaps are seeing some attenuation.

In some categories, revenue growth is more challenged which highlights a challenge in very fast growing markets : quality of revenue.

The underlying technology shifting rapidly - the announcements from Google & OpenAI this week underscore the point; buyer preferences are evolving rapidly as they start to understand the relevant applications of the technology, the costs of deployment & the value gained from different applications ; vendor pricing dynamics also play a role. OpenAI reduced prices by 50% this week driving deflation in the market.

The additional data within the report underscores AI’s growth in the last year.

“Card transaction volume for AI software services jumped 293% year-over-year compared to an increase of just 6% in overall software transaction volume during the same period.”

This growth is true across company size.

& across most industries.

There’s still quite a lot to be excited about. & there are signs to be more sanguine about long-term growth rates & valuations approaching those of overall high-growth software companies.

It’s also important to note that this spending data is just credit cards. For many of these companies whose larger accounts pay via purchase order, the patterns may be quite different.

¹ I’m annualizing the Q1 changes to impute the 2024 estimated change.

2024-05-16 08:00:00

On Thursday May 30th at 9:30 Pacific time, Office Hours will host Benn Stancil.

Benn co-founded Mode, a analytics platform acquired by ThoughtSpot. He writes about the future of data & analytics on his blog & asks deep questions about core tenets of software including Do software companies actually have good margins? & Disband the analytics team.

With a unique lens on data & software, Benn will share his views on some of these bigger topics including BI’s third form.

During this Office Hours, Benn & I will talk about :

• what is BI’s third form & why it matters
• the role of the analytics team in the world of AI where the there are more shovels than gold
• margin structures for software companies : why might they be less attractive than they seem

If you’re interested to attend, please register here. As always, submit questions through the registration form & I’ll weave them into the conversation.

I look forward to welcoming Benn to Office Hours!

2024-05-13 08:00:00

The Theory AI Index of publicly traded companies continue to outperform more classical software companies since we published the index earlier this year.

In addition to the delta in absolute multiple, is there a difference in the factors that drive valuation?

Across the five most important metrics, AI companies’ valuation correlates isn’t that much different than non-AI companies.

Revenue growth, sales efficiency, & cash flow are all relatively similar. So is net income margin, although the AI correlate is about half. The only one that stands out is gross margin, where the correlate is mildly negative.

Let’s look at the stack rank.

Revenue growth & sales efficiency are top correlates for valuation. Meanwhile the profitability metrics of cash flow margin, net income margin & gross margin are less important.

There are some meaningful differences in the order of magnitude to which these correlates correspond to the forward multiple, but that’s very likely explained by broader differences between these two categories underlying business characteristics.

So while AI companies do fetch a premium, public market investors view them as software companies, even if they do have robots furiously typing away inside their data centers.

2024-05-09 08:00:00

When a person asks a question of an LLM, the LLM responds. But there’s a good chance of an some error in the answer. Depending on the model or the question, it could be a 10% chance or 20% or much higher.

The inaccuracy could be a hallucination (a fabricated answer) or a wrong answer or a partially correct answer.

So a person can enter in many different types of questions & receive many different types of answers, some of which are correct & some of which are not.

In this chart, the arrow out of the LLM represents a correct answer. Askew arrows represent errors.

Today, when we use LLMs, most of the time a human checks the output after every step. But startups are pushing the limits of these models by asking them to chain work.

Imagine I ask an LLM-chain to make a presentation about the best cars to buy for a family of 5 people. First, I ask for a list of those cars, then I ask for a slide on the cost, another on fuel economy, yet another on color selection.

The AI must plan what to do at each step. It starts with finding the car names. Then it searches the web, or its memory, for the data necessary, then it creates each slide.

As AI chains these calls together the universe of potential outcomes explodes.

If at the first step, the LLM errs : it finds 4 cars that exist, 1 car that is hallucinated, & a boat, then the remaining effort is wasted. The error compounds from the first step & the deck is useless.

As we build more complex workloads, managing errors will become a critical part of building products.

Design patterns for this are early. I imagine it this way :

At the end of every step, another model validates the output of the AI. Perhaps this is a classical ML classifier that checks the output of the LLM. It could also be an adversarial network (a GAN) that tries to find errors in the output.

The effectiveness of the overall chained AI system will be dependent on minimizing the error rate at each step. Otherwise, AI systems will make a series of unfortunate decisions & its work won’t be very useful.

2024-05-09 08:00:00

The fastest growing category of US venture investment in 2024 is AI. Venture capitalists have invested $18.3 billion through the first four months of the year.

At this pace, we should expect AI startups to raise about $55b in 2024.

AI startups now command more than 20% share of all US venture dollars across categories, including healthcare, biotech, & software.

In the preceding eight years, that number was about 8% per year. But after the launch of ChatGPT in 2022, there’s a marked inflection point.

Some of this is new company formation, & there has been a significant amount of seed investment in this category. Another major contributor is the repositioning of existing companies to include AI within their pitch.

Over time, this share should attenuate, primarily because every software company will have an AI component, & the marketing effect for both customers & venture capitalists, will diffuse.

Not surprisingly, investors have concentrated total dollars in a few names, with the top three companies accounting for 60% of the dollars raised. Power laws are ubiquitous in venture capital & AI is no exception.

2024-05-01 08:00:00

Amazon announced their earnings yesterday. Like Microsoft & Google, Amazon’s Web Service business is seeing a surge of growth, up from 13% annual to 17% annual growth (16% when excluding the leap year).

Aside from the overall growth of these clouds increasing, the massive investment in CapEx data centers, power plants, and GPUs is stunning.

These are not one-time investments, but part of a broader trend that started to occur after the introduction of GPT 3 in mid-2020 Amazon was the first to invest significantly.

Google and Microsoft would wait another two years to replicate a similar level of investment.

Each of these businesses are large enough to justify it. Here are some highlights from Amazon’s earnings :

“We see considerable momentum on the AI front where we’ve accumulated a multibillion-dollar revenue run rate already.”

Over time, we should expect Amazon and Google, amongst others, to start to compete with Nvidia GPUs, offering their own which should meaningfully improve margins.

“We have the broadest selection of NVIDIA compute instances around, but demand for our custom silicon, Trainium and Inferentia, is quite high given its favorable price performance benefits relative to available alternatives. Larger quantities of our latest generation Trainium2 is coming in the second half of 2024 and early 2025.”

And those margins are increasing for the clouds, which should catalyze more companies, especially the largest spenders, to think about managing their own infrastructure. 8 percentage points increased margins in a quarter is titanic.

“AWS margins increased 800 basis points sequentially off Q4”

If we needed any reminder, Amazon Web Services is now a $100 billion runway business, growing 17% a year, or adding $150b in market cap per year at a 9x multiple.

“Moving to AWS. Revenue was $25 billion, an increase of 17% year-over-year, and AWS is now a $100 billion annualized revenue run rate business. "

2024-04-29 08:00:00

What drives the acquisition market of startups? It’s the big deals.

In the last decade, the total number of venture backed software M&A by count has remained relatively constant. The black line shows the linear trend across US venture backed companies with disclosed values of $50m or more.

The average & median counts by year total 58 & 55 respectively.

If there are any increases, they tend to be in the bigger acquisitions of $500 million or more - although the sample size there is sufficiently small to conclude the trend is significant.

Nevertheless, there are huge differences between the total value created by software M&A annually. The least productive year produced $3.25b in M&A value and the most productive $59.7b, a 18.4x swing.

Multi-billion dollar acquisitions, the blue bars, are the largest contributors to this swing. In 2014, 2016, 2020, 2021, these big mergers drove the figures into the tens of billions.

It’s no surprise that in those years, the biggest acquisitions accounted for more than 53% of dollars on average.

The relatively constant drumbeat of smaller acquisitions provides liquidity essential for the venture capital ecosystem, both for venture capital firms to recycle and reinvest and also for founders to generate significant outcomes and potentially try again.

However, as the VC industry has 10xed in the last decade, more large M&A will become essential to sustain great multiples for VCs.

2024-04-26 08:00:00

If it wasn’t clear before, AI is the single biggest revenue driver in cloud.

Microsoft’s Azure is winning share directly from Amazon. In Feburary, Microsoft grew 2% & Amazon lost 2%. Google is also taking share - 1% in the last year. A one percentage point share shift represents about $750 million of spend or about $5b in market cap.

“We’re fundamentally a share taker there because if you look at it from our perspective, at this point, Azure has become a port of call for pretty much anybody who is doing an AI project”

“More than 65% of the Fortune 500 now use Azure OpenAI service.”

Much of the AI spend is at the enterprise, where a 50% reduction in customer support cost or a 75% increase in engineering capacity filters billions to the bottom line.

“The number of $100 million-plus Azure deals increased over 80% year-over-year, while the number of $10 million-plus deals more than doubled. "

Copilot growth implies about a $500 million ARR product growing 35% QoQ - tripling annually.

“We now have 1.8 million paid subscribers with growth accelerating to over 35% quarter-over-quarter…more than 90% of the Fortune 100 are now GitHub customers, and revenue accelerated over 45% year-over-year. "

The rest of Azure is still growing in a nice clip of 24% annually. But AI added 7 percentage points or $4b to the revenue, a 31% growth rate for a $4b business means adding $1.2b of revenue in a year.

“7 point lift to Azure growth from AI”

Combined with the Copilot business, Microsoft has roughly $4.5b-5b AI product suite.

This trajectory shows no signs of slowing down. The company is investing $50b of capex this year into additional capacity because AI demand exceeds supply.

“Currently, near-term AI demand is a bit higher than our available capacity…In Azure, we expect Q4 revenue growth to be 30% to 31% in constant currency”

Even more impressive, if illusive to measure, is the cross-selling ability of AI across Microsoft customers. The net dollar retention figures must be world class because Microsoft is largely selling to existing customers. NDRs of 300-400% are not unlikely.

Vector databases :

“So AI projects obviously start with calls to AI models, but they also use a vector database. In fact, Azure Search, which is really used by even ChatGPT, is one of the fastest growing services for us. We have Fabric integration to Azure AI and so – Cosmos DB integration.”

The Fabric data platform :

“Fabric now has over 11,000 paid customers, including leaders in every industry from ABB, EDP, Energy Transfer to Equinor, Foot Locker, ITOCHU and Lumen; and we are seeing increased usage intensity.”

AI in other applications:

30,000 Organizations across every industry have used Copilot Studio to customize Copilot for Microsoft 365 or build their own, up 175% quarter-over-quarter.

There’s mention of more within the transcript, but I think the point comes across.

Within the Google transcripts, One of the more interesting data points is that generated search actually increases search volumes :

Most notably, based on our testing, we are encouraged that we are seeing an increase in search usage among people who use the new AI overviews as well as increased user satisfaction with the results.

Also, the company’s reach across consumers is staggering :

We have 6 products with more than 2 billion monthly users, including 3 billion Android devices. 15 products have 0.5 billion users

AI is accelerating growth in a $75b market, 20 year old market. If these growth rates aren’t a sign of anything to come, the amount of market cap creation in front of us will a garguantuan figure.

2024-04-25 08:00:00

Enterprises spend more on security but aren’t benefitting from the extra spend. Palo Alto Networks’ customers who buy security across 3 platforms spend more than 40x those that secure just one.

Despite those dollars…

“Adding incremental point products is not necessarily driving a better security outcome for them.” - Nikesh Arora, CEO of Palo Alto Networks

The average enterprise uses upwards of 70 security products. Many of these products produce alerts identifying phishing emails or network access issues or odd device behavior.

The security operations center (SOC) reviews these alerts. Estimates suggest the typical team receives between 5,000 & 11,000 alerts per day - every day. Worse still, every SOC team is short-staffed.

No wonder these teams can’t keep up with the deluge. Fewer than 10% of alerts are ever reviewed. Many security breaches could have been detected earlier with broader coverage.

Dropzone provides AI SOC analysts that never sleep.

Leveraging the power of LLMs, Dropzone’s analysts collect alerts, fetch relevant information from other systems, and then triage alerts. These agents also inform copilots that empower security analysts to ask questions of their environment leveraging the insight of the AI analysts.

RSA selected Dropzone a top 10 Finalist in the Innovation Sandbox.

We’re thrilled to support Edward & the Dropzone team in fulfilling their mission. As Senior Principal Scientist, Edward architected the AI systems at ExtraHop, a network security company acquired for $900M.

Our partner Andy wrote in greater depth about our vision for the company here.

And if you’re curious to test it on your own, you can test drive it here.

2024-04-24 08:00:00

Snowflake announced Arctic, their open 17b model. The LLM perfomance chart is replete with new offerings in just a few weeks. Overall knowledge performance is asymptoting as expected. It’s hard to discern the most recent dots.

One thing stands out from the announcement - the positioning of the model.

“The Best LLM for Enterprise AI”

Snowflake focuses on the model’s enterprise performance : SQL generation, code completion, & logic. This push will be echoed by others as models start to specialize.

When Databricks/Mosaic launched DBRX, their announcement focused on openness, training efficiency, & inference efficiency.

This is a shift from the general-purpose marketing of LLMs like GPT-4, Llama3, & Orca. Arctic & DBRX are B2B models.

With the MMLU general knowledge (high school equivalency) score now asymptoting, buyers of models will begin to care about different attributes. For a data-focused customer base, SQL generation, code completion for Python, & following instructions matter more than encyclopedic knowledge of Napoleon’s doomed march to Moscow.

The push to smaller models is another form of differentiation. Smaller models are much less expensive to operate : two orders of magnitude or more.

Meta’s most recent small model, Llama3 8b parameter, demonstrates small “over-trained” models can perform equally, as well as its larger brothers.

This is all good news for founders & users : more models, more choice, better performance across domains, lower costs for training, tuning, & inference. If the recent weeks are any indication, we should expect further flurries of advances & specialization.